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Physical systems with many degrees of freedom can often be understood in terms of transitions between a 
small number of metastable states. For time-homogeneous systems with short-term memory these transitions 
arc fully characterized by a set of rate constants. We consider the question how to extend such a coarse- 
grained description to non-stationary systems and to systems with finite memory. We identify the physical 
regimes in which time-dependent rates arc meaningful, and state microscopic expressions that can be used to 
measure both externally time-dependent and history-dependent rates in microscopic simulations. 



I. INTRODUCTION 



Physical systems with a large number of degrees of 
freedom and complicated dynamics can be understood 
in much simpler terms, if they exhibit a small number 
of metastable states. A natural coarse-grained descrip- 
tion is then found in terms of residence times in each 
of the states, joined by nearly instantaneous transitions 
between them. 

This idea has been widely used in the context of ther- 
mal equilibrium systems that are characterized by a clear 
separation of time scales between fast intra-state dy- 
namics and slow global relaxation (e.g. 1 ), giving rise to 
Markov State Models, see e.g. 2 . Here transitions are 
Poissonian, i.e. they happen with uniform propensity 
(probability per unit time) ; equivalently, the waiting time 
intervals are uncorrelated and exponentially distributed. 
Each transition is described by a single number, the rate 
constant. 

In this article we are interested in systems that 
switch between two metastable states rarely and rapidly, 
but with a time-dependent propensity. Such time- 
dependence may be caused by an external force driv- 
ing the system. Alternatively it may also arise due to 
the presence of internal degrees of freedom that do not 
fully equilibrate on the macroscopic time scale of inter- 
est, giving rise to non-Markovian macroscopic switching 
dynamics between the metastable states. We consider 
both equilibrium and non-equilibrium systems, in the 
sense that their unperturbed microscopic dynamics may 
or may not be time-reversible. 

Clearly, in such time-inhomogeneous or non-Markovian 
systems the concept of a rate constant is inadequate. 
In this article we investigate in what physical regimes a 
time-dependent rate function or a history-dependent rate 
kernel may instead be meaningfully defined, and state the 
corresponding phenomenological rate equations. We then 
address the question how macroscopic rate functions or 
rate kernels can be defined in terms of microscopic cor- 
relation functions; we give microscopic expressions and 
show how they can be used to measure these generalized 
rates in computer simulations. 



II. OVERVIEW 

We consider systems whose coarse-grained, macro- 
scopic dynamics can be described as switching between 
two macroscopic states A and B. The macroscopic states 
are assumed to be metastable in the sense that the mean 
waiting time between the switching events is much longer 
than the duration of the switching event itself. This 
makes it possible to partition the phase space into the 
metastable regions A and B, and a transition region C 
separating them, such that 

TC < TAB, T B A, (1) 

where tq is a typical duration for a traversal of C (a 
switching event), and tab,t~ba are the mean waiting 
times for switching in the forward and backward direc- 
tion, respectively. This separation of time scales provides 
the justification for coarse graining the system as a two- 
state system, switching rarely but rapidly between the 
states A and B. 

We are interested both in equilibrium systems that 
are microscopically reversible and in non-equilibrium sys- 
tems with dissipation of energy. If the system is in sta- 
tionary state, the stability of the states A and B and the 
forward and backward probability fluxes, qAB and qBA, 
are constant in time. Nonetheless, if the system has mem- 
ory, the switching propensities will be history-dependent. 
Furthermore, out of stationary state, the stability of the 
states and the dynamics of switching between them will 
change in time. In both of these situations, rate constants 
kAB and ksA do not adequately describe the system, and 
a time-dependent generalization of the rate constant con- 
cept is called for. 

Following Chandler in his derivation of rate constants 
in equilibrium systems 3 , we imagine that from experi- 
mental observations we know a certain phenomenolog- 
ical expression for the macroscopic switching dynamics 
to be valid, on times much larger than a certain macro- 
scopic time resolution At. This macroscopic expression 
is the rate equation. The goal is then to derive a micro- 
scopic expression for the dynamics of the system that is 
consistent with the rate equation, and allows us to mea- 
sure the macroscopic rates in a microscopic simulation. 
We will thus not derive a macroscopic expression from 
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microscopic principles; rather, we will assume that the 
system obeys a given rate equation, and use this descrip- 
tion as a starting point for the derivation of microscopic 
expressions for time-dependent rate functions 3 . 

The propensity of the system to switch from one 
macroscopic state to another at time t may depend on 
the current macroscopic state of the system only, in which 
case the system is Markovian. More generally, it may de- 
pend also upon the macroscopic history, i.e. the sequence 
of states visited in the past. For such a non- Markovian 
system, the switching propensity could depend on the 
time that has passed since the last switching event, but 
it is also conceivable that it depends on the system's dy- 
namics prior to the last switching event. Indeed, all in- 
formation about the history of the macroscopic switching 
dynamics is contained in the sequence of switching times. 
It is thus natural to capture the history dependence of 
the switching dynamics by writing the time evolution of 
the system in terms of the times of all previous switching 
events: 

^- t P B (t; t', t", . . . ) = k AB {t\t', t", . . . )P A (t; t>, t", ...) 

-k BA {t\t',t",...)P B (t;t',t",...), 

(2) 

In this Master equation, P A (t; t', t" , . . . ) denotes the joint 
probability to be in A at time t and to have switched for 
the last time within (t' , t'+dt') (i.e. from B to ^4), for the 
second-to-last time in (t" , t" + dt") (from A to B), and so 
on. Moreover, k AB (t\t' , t" , . . . )dt is the conditional prob- 
ability to leave A in the time interval (t, t + dt) given that 
the sequence of switching times was t' , t" , . . . . An anal- 
ogous equation holds with A and B interchanged. Im- 
portantly, this macroscopic phcnomcnological rate equa- 
tion defines the general rate kernels k AB (t\t' , t" , . . .) and 
k BA (t\t' ,t" , . . .). An equation for the switching dynam- 
ics at time t can be obtained from Eq. 2 by integrating 
over the switching times prior to t, 

Ap B ( t ) = f lp B ( t ; t', t", . . . )d*'di" .... (3) 

at Jt>t'>t">... at 

Since the last, second-to-last, . . . switches are each 
unique events, no over-counting occurs here. No approx- 
imation has been made up to this point; note however 
that 3 is not in general a closed equation for P A , B (t). 
The lower integration limit depends upon the experimen- 
tal setup and will be discussed in more detail below. 

While Eqs. 2 and 3 describe the switching dynamics 
of an arbitrary two-state system, they cannot be solved 
in general. They are also too detailed, requiring full in- 
formation about the macroscopic history. At this point 
the experiment has to inform us about the most use- 
ful and meaningful phcnomcnological model, that is, the 
minimal model that captures the macroscopic switching 
dynamics of the system. More specifically, the experi- 
ment has to reveal whether the switching propensities do 
indeed depend on the current time t and the previous 



switching times t',t", . . . . The decision about the type 
of macroscopic expression that describes the experiment 
best, will then depend both on the physical properties 
of the system under consideration, and on the time res- 
olution and the measurement uncertainty of the experi- 
ment. Below, we will discuss a number of stationary and 
non-stationary situations for which Eqs. 2 and 3 can be 
simplified, and for which microscopic expressions for the 
macroscopic rate constants can be found. 

We now turn to the microscopic characterization of 
the system. We consider an ensemble of trajectories that 
start at t = at state space points xo, following a spec- 
ified initial phase-space distribution function /j(xo) 4 . If 
the dynamics obey detailed balance and microscopic re- 
versibility, and if p(x ) is the canonical distribution, then 
this is an equilibrium ensemble. We may instead also 
consider the relaxation towards thermodynamic equilib- 
rium, starting from a non-equilibrium p(x ). Alterna- 
tively, the ensemble could be that of a non-equilibrium 
system that does not obey detailed balance and micro- 
scopic reversibility. Again, the initial condition could 
either be in stationary state, or out of stationary state, 
in which case the non-equilibrium system may relax back 
to a stationary state. Finally, the system (time-reversible 
or not) may never reach a stationary state over the time 
course [0, T] of the experiment. This can happen if relax- 
ation is slower than T, or if the system is driven exter- 
nally via some protocol 4>(t), for instance, proteins that 
are unfolded under the influence of an external force. 

To derive microscopic expressions that are consistent 
with the macroscopic rate equations, Eqs. 2 and 3, we 
first need to define functions h A (x t ) and h B (x t ) that in- 
dicate whether the system with configuration x t at time 
t is in state A or B. It is useful to define these character- 
istic functions in terms of an order parameter q(xt) that 
serves to measure the progress of the transition between 
the states A and B: 

h A (x t ) = 9[q A -q(x t )], (4) 
h B {x t ) = 6[q(x t ) - q B }. (5) 

Here, 9 is the Heaviside step function. With these 
definitions, the system is considered to be in A when 
q{x t ) < q A , in B when q(x t ) > q B , and in the transition 
region C otherwise. Moreover, we have the relation 

h A + h B = 1, if q A = q B = q* . (6) 

The same relation holds effectively if q A < q B and the 
occupancy of the transition region C is low for all times. 

The principal idea is now that the macroscopic rates 
k AB , BA (t) can be derived from the behavior of the mi- 
croscopic correlation function 

C(t) = J dx dx t p(x )p(x t \x )h B (x t ) 

= (h B (x t )) (7) 

Here, p(x t \xo) is the probability that the system is in 
state Xt at time t given that it started in state xq at 
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time zero; (...) denotes an average over the ensemble of 
trajectories that start in xq, following the phase-space 
distribution p(xq). The correlation function C(t) gives 
the probability that the system is in state B at time t 
given that it started from the initial distribution p(xo), 
which may or may not be chosen to fully reside in A. The 
transition rates are derived from the flux into B and out 
of B, and are thus related to the time derivative of C(t): 



expressions for the macroscopic rate constants, we then 
consider the correlation function 



C(t) = (h B (t)), 



(8) 



where we use the shorthand notation h B (t) — h B {%t)- 
The task at hand is to rewrite this microscopic expres- 
sion such that it can be identified with the macroscopic 
expressions of Eqs. 2 and 3. This is carried out below for 
a number of different classes of systems. In each case, 
we start with the macroscopic rate equation, and then 
derive a microscopic correlation function from which the 
macroscopic rates can be obtained. 



III. MARKOV SYSTEMS 

In this section we consider systems that exhibit Marko- 
vian macroscopic switching dynamics. By this we mean 
that it is known from experiment that there exists a 
macroscopic time resolution At, on which the propensity 
to switch between the two macroscopic states A and B 
at time t is independent of the history of the macroscopic 
switching dynamics for earlier times t' < t. In this case 
the rate kernels are independent of the previous switching 
times, yet may depend on the current time if the system 
is not time-homogeneous: k xx (t\t', t", . . . ) = k xx (t), 
where XX = AB or BA. We immediately conclude that 
the macroscopic rate equations Eqs. 2, 3 reduce to 



dt 



P B (t) = k AB (t)P A (t) - k BA (t)P B (t) 



(9) 



with rate functions k AB (t) , k BA (t) ■ This equation to- 
gether with its counterpart for P A forms a closed set of 
equations for P AtB . We remark that the form of Eq. 9 can 
always be obtained trivially from Eqs. 2, 3 by marginal- 
izing. However, the resulting expression will then in gen- 
eral only be valid for a particular experiment. A Markov 
system is characterized by the property that Eq. 9 with 
a fixed set of rate functions k xx (t) correctly describes 
multiple experiments, differing in their initial conditions 
and in their history before t = 0. 

The fact that memory is lost on the macroscopic time 
scale At implies that the system locally equilibrates in 
the macroscopic states A and B on time scales t a and 
t b , respectively, that are shorter than At. We can then 
identify a transient time scale 

Ttrans = maxf^ , T B , T C } < At < min{TAB, T BA } (10) 

which is the microscopic memory time of the system. 

It is instructive to restrict the attention to trajectories 
that happen to start in A at t = 0. To derive microscopic 



C(t) = (h B (t)) Ao , 



(11) 



where the subscript A indicates restriction of p(xo) to 
this subensemble. We now rewrite this expression such 
that its terms can be identified with the macroscopic 
quantities appearing in Eq. 9. Taking q A = q B = q* 
and letting h x (t) — h x (x t ), we can insert h A (t — At) + 
h B (t - At) = 1 and h A (t + At) + h B (t + At) = 1 into 
Eq. 11; the importance of At will become clear shortly. 
This yields 

C(t) = (h A {t - At)h B (t)h A (t + At)) Ao 
+ (h A (t - At)h B (t)h B (t + At)) Ao 
+ (h B {t - At)h B (t)h B (t + At)) Ao 
+ (h B (t - At)h B (t)h A (t + At)) Ao . (12) 

We now condition on the state prior to a transition, by 
multiplying and dividing the first two terms by (h A (t — 
At)) Ao and the last two terms by (h B (t — At)) Ao . This 
gives 

C(t) = (h A (t - At)) Ao (h B (t)h A (t + At)) A() . At _ &t 
+ (h A (t - At)) Ao (h B (t)h B (t + At)) Ao , At _ At 
+ (h B (t - At)) Ao (h B (t)h B (t + At)) Ao Bt &t 
+ (h B (t - At)) Ao (h B (t)h A (t + At)) Ao , Bt _ At (13) 

The average (. . . )A ,x t _ At runs over trajectories that 
start in A at time t = and are in X = A, B at time 
t - At. 

Since the system looses memory on the transient time 
scale Tt rans , the memory fo the initial state is lost for 
(t-At) > T trans , so that (. • . )x t _ At ,A c = (■••)x t _ At - In 
this regime Eq. 13 becomes 

C(t) = (h A (t - At)) Ao (h B (t)h A (t + At)) At _ At 
+ (h A (t - At)) Ao (h B (t)h B (t + At)} At _ &t 
+ (h B (t - At)) Ao (h B (t)h B (t + At)) Bt _ &t 
+ (h B (t-At)) Ao (h B (t)h A (t + At)) Bt _ At . (14) 

We now rewrite the first and third term in this equa- 
tion, which describe trajectories that re-cross into their 
original state X — A,B after a time 2At: 

C(t) = ( ^ (t "_ A ^° <M* - *t)h B (t)h A (t + At)) 
+ (h A (t - At)) Ao (h B (t)h B (t + At)) At _ &t 

+ (h B (t - At)) Ao {h B (t)h A (t + At)) Bt _ At 
= j AA (t;At)+j AB (t;At) 

+ ]BB (t;At)+j BA (t;At). (15) 
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Figure 1. A visual interpretation of the term jaa = {h,A(t — 
At)hB{t)h A {t+At)) a ofEq. 12. In microscopically reversible 
systems (a), for each trajectory (blue) there exists an identical 
but time-reflected trajectory (red). Consequently, jaa = for 
all values of At. In non-equilibrium systems that are memo- 
ryless and in stationary state (b), for each trajectory (blue) 
there exists a time-shifted trajectory (green) which cancels 
the contribution to j AA , so that jaa = as well, irrespective 
of time-reversibility. In systems that are not time-reversible 
and remain correlated over At (c), time-shifted trajectories 
may fail to contribute to jaa, as is the case for the trajectory 
shown in green. Here cancellation cannot occur and jaa 0. 
See Appendix A. 



If the system obeys microscopic reversibility, and if the 
initial condition p(xg) coincides with the canonical dis- 
tribution, then (...) denotes an equilibrium average. In 
this case the recrossing terms 

j AA (t,At) = 0, j BB {t,At) = (16) 

for all times t and for all values of At. This 
can be seen by noting that the equilibrium average 
(h x (xt-At)hB(xt)hx(x t +At)) changes sign under time 
reversal and therefore vanishes in thermodynamic equi- 
librium. Out of equilibrium, the recrossing terms are 
non-zero in general. However, under the additional con- 
dition that over the given time scale At the system is ap- 
proximately time-homogeneous and memory less, Eq. 16 
can be shown to hold nonetheless (Appendix A). Fig. 1 
gives a graphical account of these observations. 

Accepting Eq. 16 for the moment (this will be justified 
case-by-case below), Eq. 15 reduces to 

C(t) = (h A (t - At)) Aa (h B (t)h B (t + At)) At _ &t 

+ (h B {t-At)) Aa (h B (t)h A {t + At)) Bt _ At (17) 

We can now relate the microscopic Eq. 17 to the macro- 
scopic Eq. 9 by identifying (h B (t — At)) Ao = P B (t — At), 
(h A {x t -At))A = P A {t-At) and C(t) = dP B {t)/dt. This 
yields 

^P B (t) = P A (t- At) (h B (t) h B (t + At)) At _ &t 

+ P B {t - At){h B {t)h A {t + At)) Bt _ At , 

~P A (t){h B (t)h B (t + At)) At _ At 

+ P B (t)(h B (t)h A (t + At)) Bt _ &t , (18) 



where the approximate equality results from the fact that 
At is the time resolution of the macroscopic descrip- 
tion. Comparing this expression with the macroscopic 
rate equation Eq. 9, we deduce that 

k AB (t) = (h B (t)h B (t + At)) At _ At , 

k BA {t) = -{h B (t)h A (t + At)) Bt _ At . (19) 

In the next section, we recapitulate for completeness 
the classical scenario of a Markov system that relaxes 
towards stationary state starting from a non-stationary 
distribution 3 . In the subsequent section we consider 
Markov systems that are driven via an external protocol. 
In both cases, we will first consider the phenomenologi- 
cal rate equation, and then demonstrate that the macro- 
scopic rate constants k AB (t), k BA (t) are given by the mi- 
croscopic expressions on the right-hand side of Eq. 19, 
provided At is chosen carefully, as discussed below. 

A. Time-homogeneous Markov systems 

We imagine that the Markov system has been driven 
via some protocol into an arbitrary initial condition p(xo) 
at t = 0, and consider its relaxation towards stationary 
state. We further imagine that experiments have revealed 
that the phenomenological rate equation, Eq. 9, can be 
reduced to 

^-P B (t) = k AB P A (t)-k BA P B (t), (20) 
at 

with rate constants k AB and k BA ; thus k AB is the (con- 
stant) propensity that the system switches to B, given 
that it is in A. The transitions between the states A and 
B on the macroscopic time scale At are described by a 
time-homogeneous Markov process. We note that even 
a system with time-homogeneous microscopic dynamics 
may fail to obey Eq. 20, if the system starts from a highly 
non-equilibrium initial condition; indeed, the experiment 
has to reveal whether the assumption of constant rates 
actually holds. 

The solution of Eq. 20 for a system that starts in state 
A at time t — is given by 

P B (t)=P^(l-e-*/ T '™), (21) 

where r rxn = {k AB + kj^s)^ 1 is the macroscopic re- 
laxation time of the switch and Pg? is the probability 
of being in B in the stationary state, given by Pg 3 = 
k AB /(k AB + k BA ). 

Following the discussion above in this special case, we 
first need to establish that the recrossing fluxes vanish, 
Eq. 16. This holds whenever the system is both memo- 
ryless and time-homogeneous for time differences larger 
than At. These conditions are met in the present case, 
because Eq. 20, which is assumed to describe the ex- 
periment, implies that on the time scale At the system 
switches between A and B in a memoryless fashion with 
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constant rates. It then follows that the rates are given 
by Eq. 19: 

k AB = (h B (t)h B (t + At)) At _ &t , 

k BA = -{h B {t)h A {t + At)) Bt _ &t . (22) 

While k AB and k BA on the left-hand side are independent 
of time, the expressions on the right-hand side appear to 
depend on t and At. 

We first discuss the dependence on t. In writing down 
our macroscopic rate equation, we have assumed that the 
system switches between the states A and B with con- 
stant rates. This presupposes that the system relaxes 
inside the basins of A and B in between the switching 
events faster than the macroscopic time At, leading to 
Markovian switching dynamics and to loss of memory of 
the initial condition. Importantly, this assumption also 
implies that during relaxation towards stationary state, 
the probability of being in either A or B will change with 
time, but the state-space distribution within each macro- 
scopic state A and B does not change with time; by de- 
duction, the state-space distribution during relaxation, 
conditioned on being in either A or B, must be equal 
to that of the stationary distribution. In other words, 
while the ensemble brackets (...) Ao denote an average 
over trajectories that start from an arbitrary initial con- 
dition p(xo) in state A, this ensemble must, according 
to our assumption, be effectively equal to the stationary 
ensemble of trajectories, conditioned on starting in A. 

This is essentially the content of Onsager's regression 
hypothesis 5 ' 6 , which states that the relaxation of an ob- 
servable in a non-equilibrium experiment is proportional 
to the relaxation of a spontaneous fluctuation of that ob- 
servable in the equilibrium system. It relies on the idea 
that the non-equilibrium initial distribution in a relax- 
ation experiment follows a phase-space distribution that 
is similar to that of a spontaneous fluctuation in the 
equilibrium system 7 . The regression hypothesis is a well- 
known theorem for systems close to thermal equilibrium 8 , 
but generalizations also exist for relaxation of perturba- 
tions towards non-equilibrium stationary state 9-12 . For 
rare switching events in equilibrium and stationary non- 
equilibrium systems, the presence of a dynamical bot- 
tleneck for switching means that the system can rapidly 
relax inside the basins of A and B, leading to loss of 
memory of the initial condition. This implies that the 
regressions theorem also holds, with exponential time- 
dependence of the relaxation function, at least on a time 
resolution At coarser than the transient time r trans . 

We now turn to the dependence on At. Since the rates 
are constant, we can time-shift Eq. 22 to give: 

k AB = (h B {0)h B {At)) A _ At , 

k BA = -(h B {0)h A {At)) B _ At . (23) 

These expressions, which are very similar to those of 
the reactive flux method of Bennett 13 and Chandler 3 , 
hold only for a certain range of At values 7 . The 



macroscopic rate constant k AB is defined as the (con- 
stant) propensity that the system switches from A to 
B given that at that moment in time it is in A. In 
contrast, the microscopic expression (h B (0)h B (At)) A _ At 
is the propensity that the system switches from A to 
B minus the propensity that it switches from B to A 
at a certain moment in time, given that at an earlier 
time —At it was in A and at a later time At it is in 
B: (h B (0)h B (At)) A _ &t = {6[h B {<S)]h B {<S)h B {At)) A _ At - 
(6[-h B (0)](-h B (0))h B (At)) A _ &t . Indeed, the micro- 
scopic expression takes into account that the system may 
switch back and forth between the two states a number 
of times, and in fact, for At — > oo, (h B (0)h B (At)) A _ At 
becomes equal to (h B (oo))(h B (oo)) = 0. Clearly, the 
macroscopic rate constant can only become equal to the 
microscopic expression if At is chosen to be smaller than 
the typical waiting time, At <C t ab ,t ba . 

On the other hand, At cannot be made arbitrarily 
small. For At — > 0, Eqs. 23 reduce to the transition- 
state approximation 

fcH T - (9(#(?-9*)>, 

kl A T = -(e(-q)qS(q-q*)). (24) 

Transition-state theory assumes that every trajectory 
that crosses the dividing surface from A to B will end up 
in B before it returns to A on a time scale t ba . However, 
trajectories may also recross the dividing surface a (large) 
number of times before they settle in the new state B or 
the original state A. These correlated recrossings tend 
to decrease the correlation function (h B (0)h B (At)) A _ At 
as At is increased from zero. If the recrossings can be 
resolved experimentally, and if one wishes to character- 
ize them, then Eq. 20 is not an appropriate model and 
the rate constants have to be defined differently; we will 
discuss this scenario in more detail in the next section. 
Here, we have been assuming that Eq. 20 is an appro- 
priate model. In this model, the system equilibrates on 
time scales t a < At and t b < At inside the states A 
and B, before switching out of these states on a much 
longer time scale t ab and t BAi respectively. This im- 
plies that the correlation functions (h B (0)h B (At)) A _ At 
and (h B (0)h B (At)) B &t reach a plateau for At in the 
range T trans < At < t ab ,t ba . 

Figs. 2 and 3 illustrate these ideas for a particle 
performing a random walk in a piecewise-flat double- 
well potential according to Metropolis Monte Carlo dy- 
namics (e.g. 14 ). The macroscopic trajectory, shown in 
Fig. 2b, exhibits long committed episodes, but also short 
spikes caused by transient recrossings of the dividing 
surface q* = q A = q B . While the correlation function 
C(t) = (h B (t)) Ao for this system shown in Fig. 3a rises 
exponentially for macroscopic times, its time derivative 
(Fig. 3b,c) does show a sudden drop for t < Tt ra ns ~ 10; 
this is due to the rapid correlated re-crossings of the 
diving surface q* . Importantly, for t tr ans < t < r rxn 
the derivative of the correlation function exhibits a clear 
plateau. In this regime, the flux of trajectories from A to 
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Figure 2. Microscopic (a) and macroscopic (b) trajectories 
in a simple model system of diffusion over a flat potential 
barrier. A particle starts at q = —2.5 and then performs a 
random walk visiting half-integer lattice points, with reflect- 
ing boundaries at q = ±15. The particle experiences a flat 
potential barrier given by t/(<?)//csT = 30(2 — diffusive 
steps are proposed in either direction and accepted accord- 
ing to a standard Metropolis criterion. The dividing surface 
defining the states A and B is located at qA = qs = q* = 0. 




100 1000 
t [MC steps] 

Figure 3. The correlation function C(t), Eq. 7 and its time 
derivative for the model system of Fig. 2. Clearly on long 
times on the order of r rxn ~ 1800 MC steps the system re- 
laxes exponentially. For very short times t < 10, the behavior 
is dominated by correlated transient re-crossings made by tra- 
jectories on the barrier top. A plateau emerges between these 
two times, whose value C ~ 2.78 x 10~ 4 equals the rate con- 
stant kAB- 



B is constant. Indeed, the rate constant kAB is precisely 
given by the value of C{t) in this regime. 

In summary, we reiterate that we have not derived the 
macroscopic rate equation from microscopic laws. By 
positing the macroscopic rate equation Eq. 20, we have 
made the implicit assumption that the system relaxes in- 
sides the basins and looses memory of its dynamics on 



the experimental time scale At. The same assumption 
also entails that the non-stationary ensemble of trajec- 
tories is similar to the stationary ensemble of trajecto- 
ries, conditioned on starting in A. In other words, if the 
macroscopic rate equation gives an accurate description 
of the switching dynamics, then this strongly suggests 
that the Onsager regression theorem holds as well. 



B. Externally driven Markov systems 

We now consider timc-inhomogeneous Markov sys- 
tems. We imagine that from t = the Markovian system 
is under the influence of some time-dependent external 
force 4>(t) that biases it towards one state or the other. 
The macroscopic rate equation thus has the general form 
of Eq. 9, and is valid for times longer than a macroscopic 
time At as inferred from experiment. The general solu- 
tion can be written as 



P B (t) = P B (0)e 



Jo 



T P5m (*') _|_ 



k 



All 



_ rt dt" 

[t')e Jt ' w*")di', 
(25) 

where we define T rxn (i) = fcyis(^) + ksA(t). 

Examples of externally driven Markov systems include 
protein unfolding driven by a time-dependent force and 
(crystal) nucleation at a time-varying temperature. This 
class also contains systems where one (or more) of the 
degrees of freedom relaxes slowly on the time scale of a 
switching event, such as gene networks where at a given 
time a gene is turned on, which then induces the flipping 
of a genetic switch. The protein that drives the flipping of 
the switch could then be viewed as the external force (j>(t) 
that acts on the switch. We consider external protocols 
with finite band-width such that there exist a scale of 
fastest force variation and a scale of slowest variation 
T^, where is at most the duration of the experiment, 
T < T. 

What microscopic scenarios are compatible with the 
macroscopic rate law, Eq. 9? The rate law implies 
that the system looses microscopic memory beyond At, 
so that T trans < At must hold. As before, we re- 
strict our attention to systems with a time scale separa- 
tion between intrastate relaxation and global relaxation, 
Ttrans "C tab.ba, where tab and tba are the mean wait- 
ing times for switching 15 . Under these constraints, there 
still is a variety of possible relations between the time 
scales of driving and the intrinsic microscopic time scales 
of the system. 

The simplest situation is the quasi-static case: While 
the state-space distribution evolves with <p(t) in between 
the switching events, the system can locally (i.e. within 
the macroscopic states) adapt to the force, meaning 
that T trans < T0. Importantly, this quasi-static forcing 
scenario does not require the force to be slower than 
the global relaxation; t^T^ ^ tab,t~ba- We then ex- 
pect that it should be straightforward to define time- 
dependent rate functions in terms of microscopic corre- 
lation functions. 
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To derive the microscopic expressions that are con- 
sistent with the macroscopic Eq. 20, we first note that 
in this quasi-static case we may choose At such that 
Ttrans < At < T<f,, T rxn . On this time scale At the macro- 
scopic switching dynamics is memoryless and approxi- 
mately time-homogeneous, since the system adapts in- 
stantaneously to the force. Thus Eq. 16 applies and the 
expressions Eqs. 14-18 hold unchanged. We then indeed 
arrive at Eq. 19. The rates now depend on time t, but 
since the force is quasi-static, the dynamics in each At 
interval is governed only by the value of <f>. Time depen- 
dence thus enters only via <f>(t): 

K xx 

(t) = k xjt W)), if 

Ttrans 

< At < i> (26) 

Furthermore, when At is in the range Tt ra ns < At < 
Tij, , T rxn , we can expect that the correlation functions in 
Eq. 19 are constant as a function of At, in analogy to the 
time-homogeneous case. 

Fig. 4 illustrates these relations in a version of the 
barrier-diffusion toy model, which is now augmented 
with an oscillatory force with a single frequency, f(t) = 
ak B Tsin{2'Kt/T <t> ) with r = T = 400, a = 0.1. We 
choose At = 20 so that r trans < At <C r^,r rxn . This 
separation of time scales suggests that we can treat this 
time-inhomogcncous Markov system as quasi-stationary, 
so that Jaa — (Eq. 16), and time enters the rates only 
via the force. Indeed, we observe that the recrossing 
fluxes approximately vanish, and that at the times when 
the force crosses zero, the rate constant equals that of the 
time-homogeneous Markov system of Fig. 3, supporting 
this idea. 

An alternative simple scenario arises in the opposite 
case of rapid driving, where the external force variations 
are faster than the local relaxation within the macro- 
scopic states, i.e. < < r trans . We can then choose a 
macroscopic resolution At > r^, T$, T trans , so that the 
macroscopic description effectively averages both over 
microscopic correlations of the system and over the vari- 
ations in driving. In this case the macroscopic switch- 
ing dynamics is described by the phenomenological rate 
equation of the time-homogeneous system, Eq. 20, al- 
though the magnitudes of the rate constants, which are 
independent of At, will be renormalizcd by the time- 
varying force. Moreover, because the system is effectively 
time-homogeneous and switches in a memoryless fashion 
between A and B , again Eq. 16 holds, so that the rate 
constants are given by the plateau values of the correla- 
tion functions in Eqs. 23 in the regime r^, r trans < At <C 
tab, tba- 

In the previous two scenarios we were able to choose 
At to be both well-separated from the driving time scales 
and within a well-defined plateau of the derivative of the 
microscopic correlation function; this allowed us to ex- 
tract the rate as fc^s = C(At), irrespective of the pre- 
cise value of At. We now consider a scenario in which 
this is not possible. Suppose that a certain macroscopic 
time resolution At is required, which still leads to Marko- 
vian switching (T trans < At) but lies within the frequency 




time t [MC steps] 



Figure 4. Time-dependent version of the system of Fig. 2, un- 
der the influence of an external force. The system is started 
in stationary state within A and relaxes with a time depen- 
dent modulation (a; (/ia), blue, (/is), red). Fluxes defined in 
Eq. 12 (b; jab, blue, 3b a, red, jaa, black) reflect the state 
occupancies. The rate functions, Eq. 19 (c; colors as in b) 
are independent of state occupancy. At the zeros of the force 
(dotted lines) the time dependent rates coincide with the rate 
constant of the stationary system, k,AB = 2.78 x 1CP 4 (see 
Fig. 3), in agreement with a quasi-stationary description on 
the time scale of the driving. 



band of the driving, < At < T$. For instance, we may 
wish to describe a non-stationary experiment carried out 
with a given time resolution equal to At. 

In the case that the microscopic relaxation of the sys- 
tem is the fastest process, Tt ran s < as in the quasi- 
stationary case studied above, then the correlation func- 
tion does exhibit a plateau, but in a regime Tt rans < At < 
T0 below the macroscopic resolution At. While this does 
allow us to uniquely define a force-dependent rate con- 
stant as kAB{4>) = C{At), this rate constant is inacces- 
sible in experiments on the time scale At. The macro- 
scopic rate constant as measured in the experiment now 
depends on the experimental time resolution At: 

nt+At/2 

kf B (t) ~ (At)- 1 / k AB m')W- (27) 

Jt-At/2 

Alternatively, when the driving contains components 
that are faster than microscopic relaxation, < r trans , 
the system is Markovian on the time scale At but re- 
tains memory of the driving over times Tt rans - In this 
case (t^ < Ttrans < At < T$) the correlation function 
C(t) does not exhibit a well-defined plateau on any time 
scale, and, as a result, a rate constant cannot be defined 
in a unique, protocol-independent manner. It is of course 
still possible to define a rate function in terms of a first 
passage time density, as a function of the external pro- 
tocol; for analytical treatments in model systems with 



8 



simple driving protocols see for instance 16 and references 
therein. 



IV. NON-MARKOVIAN SYSTEMS 

For equilibrium systems, the two observations of rapid 
switching tq <C tab,t B a (Eq. 1) and rapid equilibra- 
tion ta <C tab',tb <C tba (Eq. 10) are often intimately 
connected. Rare events typically arise because of a sin- 
gle large free-energy barrier separating the two states A 
and B. As a consequence, the waiting time tends to be 
much longer than the switching event itself — making a 
two-state description meaningful — while it also allows the 
system to relax inside the basins of A and B in between 
the switching events — leading to memoryless switching 
on the macroscopic time scale At. In these equilibrium 
systems there is only one relevant macroscopic time scale 
r rxn , which is associated with the global relaxation of 
state occupancies. However, even for equilibrium sys- 
tems, incrtial effects may lead to correlated recrossings 17 , 
possibly giving rise to non-Markovian switching dynam- 
ics on the macroscopic time scale At; if such correlated 
recrossings are important, then the macroscopic model 
of Eq. 20 needs to be refined. 

One can readily find examples of non-equilibrium sys- 
tems, however, that can be coarse grained as two- 
state systems, but in which switching between the two 
states occurs in a non-Markovian fashion even on time 
scales that are comparable to the typical waiting times 
tab, tba- This means that while the waiting time is 
much longer than the duration of the switching event, 
there exists another time scale in the system that is com- 
parable to the waiting time for switching. 

An example of such a non-Markovian, non-equilibrium 
system is the bacterial flagellar motor. This rotary mo- 
tor can run in either clockwise or counterclockwise direc- 
tion. The transitions between these two states happen 
much faster than the typical waiting time for switching, 
suggesting a two-state description. In experiments, the 
distribution of waiting times was observed to be non- 
exponential, and the power spectrum of the switching 
dynamics featured a peak around Is -1 18 , which means 
that there is a characteristic frequency at which the mo- 
tor switches. 

These data show that the flagellar motor cannot be 
described random telegraph process in which the 

switching events are independent, and the clockwise and 
counterclockwise intervals are uncorrelated and expo- 
nentially distributed; indeed, this system cannot be de- 
scribed by Eq. 20. The data also imply that motor 
switching is coupled to a non-equilibrium process 19-21 . 
It appears that the conformational dynamics of the rotor 
proteins is coupled to the slow relaxation dynamics of the 
flagcllum, which in turn is influenced by the rotation of 
the motor, driven by a proton motive force 21 . 

To describe systems like the bacterial flagellar motor 
in which memory is important, we now consider non- 



Markovian systems that when left unperturbed, reach 
a stationary state (equilibrium or not). As before, we 
imagine that the system flips between two macroscopic 
states A and B that are metastable in the sense that the 
waiting time between the switching events is much longer 
than the duration of the switching event itself. These 
system have memory on the macroscopic time scale At 
which means that the propensity of switching between 
the two macroscopic states at a given moment in time 
depends upon the history of the switching dynamics. 

To derive microscopic expressions for the rate functions 
in the presence of memory, we start again with the phe- 
nomcnological description of the system. If the propen- 
sity to switch depends upon the macroscopic history even 
prior to the last switching event, then Eq. 2 cannot be 
simplified much further, and remains of little use. How- 
ever, it is conceivable that the switching propensity de- 
pends upon the macroscopic history only since the last 
switching event, to a good approximation. The bacte- 
rial flagellar motor is an example of such a system: when 
the experimental clockwise and counterclockwise inter- 
vals of the flagellar motor were randomly shuffled, the 
power spectrum was unchanged; moreover, the observed 
power spectrum could be reproduced from the measured 
waiting-time distributions only 18 . This strongly suggests 
that the different intervals are in fact temporally uncor- 
related, so that the switching propensity only depends 
upon the time that has passed since the last switching 
event. We refer to this as the clock-resetting scenario — 
after each switching event, the system looses memory; 
in other words, switching is a renewal process. In what 
follows we will restrict our attention to this case. We do 
not exclude at this point the presence of external driving; 
in that case the system has only a single-switch memory 
but is time-inhomogeneous. 

In the clock-resetting scenario, the phenomenological 
rate equation Eq. 2 can be simplified: 

Ap s (t) = f k A B{t\t')P A {t; t') - k BA (t\t')P B (t; t')dt'. 
Jt>t> 

(28) 

Here, Pa(*; t')dt' is the joint probability that the system 
is in state A at time t and has switched into this state 
for the last time within the earlier interval (t',t' + dt'). 
It can be decomposed as the propensity qBA(t') to enter 
A at t', times the survival probability SA(t\t') to stay in 
A until at least time t: P A (t;t') = q B A(t')S A (t\t'). 

The rate kernel kAB{t\t') is defined as the propen- 
sity that the system switches from A to B at time 
t given that it has switched into A at time t' < t 
and is still in A at t. It is given by the propensity 
q_AB{t\t') = —dtSA{t\t') that a trajectory that entered 
A at time t', switches from A to B at a later time t, 
divided by the probability that it is still in A at time 
t: k AB (t\t') = qAB(t\t')/S A (t\t') = -d t \nS A {t\t'). If 
the system is time-homogeneous, i.e. without external 
time dependence, then we have kAB{t\t') = kAB(t — t'). 
If the system is in stationary state, then furthermore 
Pe(t; t') = P B (t — t'). We note that this phenomcnologi- 
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cal model can be solved analytically 21 , see also Appendix 
C. 

Before we derive microscopic expressions for the rate 
kernels that are consistent with Eq. 28, we first discuss 
the time scales that are relevant in such a system. A 
two-state description implies that the duration of the 
switching event is much shorter that the waiting time 
for switching: tq -C tab^tba- Moreover, we expect that 
inside the states A and B most degrees of freedom relax 
quickly, within T4 and tb respectively, where ta <C tab 
and tb <C tba] we can thus identify a fast transient time 
scale Ttrans — max{Tc , ta , tb } • If these were the only 
time scales in problem, then the macroscopic switching 
dynamics could be described as that of a two-state Pois- 
son process, assuming that we choose a macroscopic time 
scale At > T trans . However, the observation that the 
macroscopic switching dynamics exhibits memory on the 
scale At, implies that there is another time scale in the 
problem, r s i ow , which is at least on the order r s i ow ^ At. 

The bacterial flagellar motor provides a concrete il- 
lustration of these ideas: while the switching dynamics 
of the motor in the presence of the flagellum is non- 
Poissonian, the dynamics in the absence of the flagellum 
is Poissonian, with constant rates, leading to exponential 
waiting-time distributions 22 . The reason for this change 
in behavior is that the relaxation dynamics of the flag- 
ellum introduces a slow time scale, which becomes com- 
parable to, and can even set, the typical waiting time 
for switching. After a switching event, the flagellum first 
unwinds, but then, driven by the rotation of the motor, 
winds up in the new direction; this leads to an increase 
in the force on the rotor proteins, which then tends to 
switch their conformation, and thereby the rotation di- 
rection of the motor 21 . 

We extend our diffusive-barrier crossing model to cap- 
ture the non-equilibrium switching of the bacterial flag- 
ellar motor, see Fig. 5. The particle diffuses in a 
piecewise constant potential with a single flat barrier 
of height 4fcBT at \q\ < 2; the state boundaries are at 
Qa = 1b = q* = 0. Whenever the particle enters one 
of the two potential wells (say, at t'), the clock is re- 
set and a restoring force ramps up over time, f c \k = 
— sgn(g)/o[l — e _ ' t_ * )' T *] with time constant = 100 
and saturating magnitude /o = 0.8/cbT 1 . The addition 
of a restoring force leads to a faster global relaxation, 
which for the chosen value of / is on the order of T&. 
Fig. 5 shows that the relaxation of this system is not 
exponential: There is an initial lag time, and relaxation 
occurs in an oscillatory fashion, because the restoring 
force introduces a characteristic time scale for switching, 
and because particle clocks are taken to be synchronized 
at t = 0. We remark that an exponential relaxation 
observed in a single experiment would not by itself im- 
ply memoryless switching: For instance, the same system 
with unsynchronized clocks at t = relaxes exponentially 
(not shown). 

To derive microscopic expressions for the rate kernels 
that are consistent with the macroscopic rate equation of 
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Figure 5. Particle diffusing with residence-time dependent 
restoring force. A trajectory (top, blue, left axis) reflects the 
restoring force (black, right axis). The relaxation of state 
occupancies when started uniformly in the B state basin 
(bottom) is oscillatory, and more rapid than without force 
(cf. Fig. 3). 



Eq. 28, we need to define an indicator function that serves 
to measure the time since the last switching event. We let 
H x (t,t') = Ut>t">t' hx{t"). By construction, H x {t,t') 
takes the value 1 if the given trajectory was in X = A, B 
without interruption from t' to t and otherwise, and 
Hx(t,t) — hx{t). One also verifies (Appendix D) that 

d t H x (t,t') = h x {t)H x (t,t'), and 
dfH x {t,t')=H x {t,t')h x {t'). (29) 

The temporal derivatives of H x (t,t') with respect to t' 
and t (Eq. 29) thus make it possible to count the last 
entry time and the first exit time out of X, respectively. 
Although the definition of H may appear convoluted, it 
is straightforward to measure in a simulation, by record- 
ing switching times along trajectories. Interestingly an 
indicator function similar to H has been employed by 
Jung et al 23 to characterize fluctuations of trajectories 
in glassy systems. 

Applying the relations of Eq. 29 and the identity Ka + 
hs = 0, we can obtain the following relation: 

h B {t) = h B {t) H x (t,t) 

X=A : B 

= h B (t)[ ]T H x (t,t )+ [ d v H x (t,t')dt' 

X=A,B 

= d t H B (t,t )-d t H A (t,t ) (30a) 
+ [ d t dt>H B (t,t') - d t d t ,H A (t,t')dt'. (30b) 

J t 

Here the probability flux in and out of B is partitioned 
into contributions from trajectories that remained in one 
state since to and then switch at t (Eq. 30a), and tra- 
jectories that had their last switching event at t' > to 
before switching at t (Eq. 30b). The latter terms allow 
us to derive correlation functions which express the rate 
kernels depending on the last switching event. 
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To derive a microscopic expression that is consistent 
with the the macroscopic equation 28, we take the en- 
semble average (...) of Eq. 30 (see also Eq. 8). The en- 
semble average is defined by the initial distribution p(x n ) 
as before (although here we do not condition on starting 
in state A), and possibly by a prescription for t < 0, as 
discussed below. We now make the identifications 

(M*)> = p B(t), (31a) 
d v {H x {t,t'))dt' = {H x {t,t')h x {t'))dt' 

= Px(t;t')dt> , (31b) 

d t d t ,{H x {t,t')) _ {h x {t)H x {t,t')h x {t')) 



d v (H x {t,t>)) (H x (t,t>)h x (t>)) 



(31c) 



where X = A, B and X = B, A. We can then rewrite the 
microscopic equation 30 in the form of Eq. 28. 



d* 



P B (t) = d t (H B (t,t )) - d t (H A (t,t )) (32a) 



+ f k AB {t\t')P A {t; t') - k BA (t\t')P B (t; t')dt'. (32b) 

The additional boundary terms Eq. 32a appear here due 
to a finite lower integration limit to- They can be treated 
in two ways. The first option is taking t — > — °°- Then 
the terms Eq. 32a vanish, because the system is not ex- 
pected to stay in a given state forever. To extract rate 
constants from a microscopic measurement, we then have 
to be able to determine the integrand in Eq. 32b for 
negative t' . This will be possible for instance if we are 
dealing with a time-homogeneous system which was pre- 
pared in a stationary state for negative times, since then 
P x (t;t') = q xx , ss S x (t - t') for all t' < 0, with a con- 
stant, stationary-state value of the influx q X x.ss> see also 
Appendix C. 

Instead, for non-stationary systems one may wish to 
explicitly account for the switching events only after 
t > without specifying the history before t < 0. Then 
one would let to — > and retain the boundary terms, 
Eq. 32a. They give a transient contribution to the macro- 
scopic evolution of the system, which summarizes how 
the system was prepared before t — 0. We can incorpo- 
rate this contribution by defining 



d t (H x (t,0)) 
(H X (t,0)) 
(H x (t,0))-- 
In this case, Eq. 32 becomes 



_ k <o 



(*). 



(33a) 
(33b) 



-P B (t) = k< B {t)P<*{t) k< A {t)P<\t) (34a) 

+ f k AB {t\t')P A {t; t') - k BA (t\t')P B (t; t')dt', (34b) 
Jo 

which has an effective Markovian part (Eq. 34a) and a 
non-Markovian (Eq. 34b) part. 



Scrutinizing the arguments above more carefully, we 
should expect that the system exhibits transient recross- 
ings of the dividing surface q* that are not persistent on 
the macroscopic time scale At. Moreover, these transient 
recrossings are not expected to lead to loss of memory: 
only macroscopic switching events, in which the systems 
fully transitions from one basin of attraction to another, 
are likely to reset the clock. This means that the identifi- 
cations made in Eqs. 31, like the TST expressions Eqs. 24, 
are erroneous in that they overcount transient switching 
events. An additional point concerns the experimental 
time resolution. It is conceivable that the experimen- 
tal resolution is sufficiently high that transient crossing 
events can be detected that do not reset the clock. If one 
wishes to describe these events, then the phenomenolog- 
ical model of Eq. 28, which only describes the crossing 
events that lead to loss of memory, needs to be modified. 
Here, we consider the case that Eq. 28 is an appropriate 
macroscopic model. The task is then to come up with a 
microscopic description that integrates over the transient 
recrossings of the dividing surface that do not reset the 
clock. 

To integrate over the transient recrossings of the diving 
surface that separates the macroscopic states A and B, 
we must modify the indicator function H xx . This can be 
achieved by introducing a 'grace interval' At over which 
such short excursions are tolerated. Therefor we define a 
relaxed version of h x as 



h x (t;At) = 0[ f h x (t')dt' - At/2]; 



(35) 



this new indicator function takes the value 1 precisely 
if in the preceding short interval, the system was pre- 
dominantly in state X; the arbitrary convention of using 
preceding and not e.g. centered intervals becomes unim- 
portant on the macroscopic scale. Observe that, as for 
hx, we have h A (t; At) + h B (t; At) = 1 (we still take the 
states to be adjacent, q A = q B = q*); the previous def- 
inition hx(t) is recovered in the limit At — > 0. Using 
this coarse-grained state indicator function, the indica- 
tor for uninterrupted residence in a state is then defined 
as before, 



H x (t,t';At)= [I h x (t";At). 



(36) 



t>t">t' 



This expression remains 1 for instance if all excursions 
from X between t' — At and t are shorter than At/2. 
By definition, H x (t, t'; At) is a monotonically decreasing 
function of t. Furthermore, Hx (t, t; At) = hx(t;At), 
and H x (t,t';At) ->• H x (t,t') as At -> 0. 

Fig. 6 shows the average uninterrupted state occupan- 
cies for the model system introduced above. While the 
state occupancy H A (t, t; At) = h A (t; At) displays the os- 
cillatory global relaxation in this system, Hx (t, t'; At) 
decays with increasing residence time t — t' . This de- 
cay is slower when At is finite due to the fact that brief 
excursions from the state are tolerated. 
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Figure 6. Uninterrupted state occupancies {HA(t, t'; At)} for 
the system from Fig. 5, for At = 0, 20 (solid and dashed lines, 
respectively). The system starts in state B and accumulates 
in A with occupancy {HA(t,t; At)) — (h,A(t;At)), shown in 
gray. Crossings back to B then decrease (i?A(t, t'\ At)}, shown 
in colors for fixed values t' = 0, 100, 200, 400 from left to right. 



With the new definitions for h and H, the equalities in 
Eq. 30 still hold. Therefore, also Eq. 32 holds unchanged 
if we make the new identifications 



(h B (t,At)) = P B (t), 
d t , (H x (t, t'; At))dt' = P x (t- 1'; At)dt' , 
dtd t >(H x (t,t';At)) 
d t ,(H x (t,t';At)) 



k xX (t\t';At), 



(37a) 
(37b) 

(37c) 



replacing Eqs. 31. 

If the system exhibits transient recrossings on a time 
scale Ttrans "C t s i ow , then k X x(t\t'; At) should become 
independent of At if we choose At to be in the range 
Ttrans < At < t s i ow . Moreover, we can expect that 
with this choice of At, the rate constant k xX (t\t'; At) 
agrees with the experimentally measured rate constant 
k xX (t\f) when the experimental time resolution is in- 
deed of the order At, such that the transient recrossings 
are not detected, while the presence of a slow time scale 
is detected. 

These observations are illustrated in Fig. 7, for the 
clock-resetting model system introduced earlier. We re- 
call that the quantity (d t >H A (t, f ; At)) = P A (t; t' ; At) 
measures the probability that the system enters A at 
time t' and stays in A at least until t; it is thus given 
by the influx propensity q BA (t'; At) at t' times the 
survival probability in A, S A (t\t';At): P A (t; t' ; At) = 
q BA (t';At)S A (t\t';At). Fig. 7a shows that P A (t,t';At) 
decreases with t — t' because S A (t\t'; At) decreases with 
t — t'. P A (t; t'; At) is non-monotonic as a function of t', 
reflecting the fact that the flux q BA (t'; At) from B into A 
shows oscillations as the system equilibrates these arise 
because the probability that the system is in B relaxes 
in an oscillatory fashion, see Figs. 5, 6. 

The rate kernel k A B(t\t'; At) is defined as the flux of 
trajectories that enter A at time t' and leave A for the 
first time at time t divided by the flux of trajectories 
that enter A at time t' and remain in A till time t, see 
Eqs. 31c, 37c. This conditioning removes the dependence 
on the start time t' if the system is time-homogeneous. 
Fig. 7b shows that k AB (t\t'; At) indeed depends only on 
the time difference: k A B(t\t'; At) = k AB (t — t'; At), so 



that we can conclude that our model system is indeed 
time homogeneous. By the same token, we may now 
average data for the same t— t' but different t', leading to 
the result shown in Fig. 7c. It is seen that k AB (t — t'; At) 
starts out with a high crossing rate, passes through a 
minimum and then rises to saturate at a constant value. 
The initial drop is due to the rapid correlated recrossings 
of the diving surface q A = q B = q*', importantly, the 
detection of these rapid recrossings is strongly suppressed 
when the grace interval At is chosen to be larger than 
the microscopic relaxation time Tt rans , At > Tt rans ~ 10, 
as seen in panel c. The subsequent minimum in k AB (t — 
t'\ At) corresponds to local relaxation of the system inside 
the basin of the new macroscopic state. The rate kernel 
remains small as long as the force that tends to flip the 
system back to the other state on a time scale is still 
low. Finally, as the switching force increases with time, 
the rate k AB increases, to reach a plateau value when the 
switching force becomes constant. Fig. 7c also shows that 
k AB (t — t';At) becomes independent of At when At > 
Ttrans ~ 10 (and t — t' > At). This is crucial: it allows us 
to define the macroscopic rate function k AB (t — t'), and it 
justifies a posteriori the two-state description with a rate 
kernel that depends on the time since the last switching 
event. 



V. DISCUSSION 

It is instructive to compare the microscopic expression 
for the rate kernel in a non-Markovian system, Eq. 37, 
with that for the rate constant in a Markovian system, 
Eq. 23. Both expressions count only those switching 
events where the system goes from one macroscopic state 
to the other over a time 2At. For a Markov system 
in stationary state, the recrossing flux j AA oc (h A (t — 
At)h B (t)h A (t + At)) = 0, as discussed above (see also 
Appendix A). That is, if we chose to count recrossings in 
the microscopic expression for the rate function instead of 
using Eq. 23, this would not change the net result except 
for additional noise. In fact, the Bennett-Chandler ex- 
pression (h B (t)) Ao = (<?(0)%(0) - <?*)%(*) - <t))/{h A ) 
also counts trajectories that come from B, cross and re- 
cross the dividing surface (a number of times) , and then 
go to B; however, their net contribution to the average 
is indeed zero 3 . 

For a non-Markovian system, (h A (t — At)h B (t)h A (t + 
At)) may be non-zero even in stationary state, due to the 
presence of memory over At. In contrast to the Markov 
case, here these transient recrossings should be explicitly 
excluded, for the reasons already mentioned above: If we 
take the macroscopic rate function to be defined as the 
propensity to switch, given the time that has passed since 
the last macroscopic switching event, then we should in- 
deed only count the truly macroscopic switching events 
in Eq. 37. 

Whether a switching event counts as a macroscopic 
switching event ultimately depends on the chosen macro- 
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Figure 7. Time-dependent survival probabilities and rate ker- 
nels for the model system from Fig. 5. (a) The joint influx 
and survival probability {d t <H A (t,t'; At)) = Pa{P, t 1 '; At) = 
qBA{t')S A (t\t') as a function of t — t', shown for different val- 
ues of t' (t'=0, 100,200,400, as indicated) and for At = 0, 10 
(solid and dashed lines, respectively). Pa(£; t'; At) is the prob- 
ability that the system enters A at time t! and still is in A at 
t. It decreases monotonically with t—t', yet more slowly when 
the grace interval At = 10 is finite; but is non-monotonic in t', 
as explained in the text, (b) The rate kernel kAB(t\t'; At) as 
a function of t — t', for different values oft', and for At = 5. It 
is seen that the rate kernels for the different times t! collapse, 
(c) The rate kernels k(t — t'; At) averaged over t', for chosen 
time lags At = 0,2,4,10,20, from top to bottom. It is seen 
that for t — t' > At and At > r tra ns ~ 10, the rate kernels 
become independent of At. The macroscopic rate kernel can 
be identified with this limiting function. 



scopic time resolution At. We can define a macroscopic 
switching event in an unambiguous manner if the time 
scale for recrossing Tt ran s, is well separated from the slow 
memory time scale t s i ow of the system, so that fc(t|t'; At) 
is independent of At in the regime r tra ns < At < r s i ow . 
It may happen that the time scales Tt ra ns and t s i ow are 
not well separated; then, to match the experimentally 
measured rate constant, it becomes critical to tune At 
exactly to the experimental time resolution. However, 
the rate kernel denned in this way will then depend on 
the details of the experiment at least over some range 
of residence times t — t', which limits its use as a de- 
scription of intrinsic properties of the system. Thus in a 
non-Markovian system described by Eq. 28, a time scale 
separation between rapidly decaying transient memory 
and a well-defined slow memory time scale is again es- 
sential. 

In this context, it is noteworthy that we have used con- 



ventional indicator functions /iA(t) and /is(t), instead of 
the indicator functions h^(t) and hg(t) as introduced by 
Van Erp and coworkers 24 . The latter are denned such 
that hj^{t) is 1 if the system was more recently in A than 
in B, and vice versa. The advantage of those indicator 
functions is that by a judicious choice of the dividing sur- 
faces qA and qs, only the macroscopic switching events 
are counted, i.e. the crossing events where the system 
goes from one basin of attraction to the other 24 . If a clear 
separation exists between the transient time scale and 
the slow time scale, and macroscopic switching events 
can thus be defined uniquely (as discussed above), then 
the indicator functions Ha,b are useful also in the frame- 
work presented here. For example, these indicator func- 
tions would then obviate the need to introduce a grace 
interval, as we have done for the non-Markovian system. 
The advantage of the conventional indicator functions 
is, however, that the corresponding correlation functions 
will inform us whether there exists such a clear separa- 
tion of time scales: if so, the correlation functions exhibit 
a plateau, from which the macroscopic rate constant can 
be obtained. 

Finally, now that we have derived microscopic 
expressions for macroscopic rate constants for both 
time-inhomogeneous Markov systems and non-Markov 
systems, the question arises whether these expressions 
can also be evaluated efficiently in a computer simula- 
tion. In the accompanying paper 25 we present a new 
numerical technique that makes this possible. 
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Appendix A: Vanishing recrossing fluxes 

We consider the terms jxx{t) = {hx{t — 
At)fiB{t)hx{t + At)) appearing in Eq. 15, where X — 
A,B. Noting that (a) for qA — qB we have /i_e(t) + 
h>A{t) = 0, we can restrict our attention to the quantity 
j(t) = (h x (t-At)hx(t)h x (t + At)). 

Assuming memory loss over time intervals At (b) , and 
regarding t ± = t ± At as fixed, we can factor the joint 
probabilities as 

j(t) = d t (h x (t-)h x (t)h x (t+)) 

= d t ((h x (t-))(h x (t))x t Ahx(t + ))x t ) 

= (hx(t-))d t (h x (t))x t .(hx(t + ))x t - (Al) 

We further assume (c) that the system is time- 
homogeneous over the time At, in other words, any ex- 
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ternal driving, or background relaxation of slow degrees 
of freedom, happens much slower. Then we may rewrite 
the conditional probabilities as functions of the time dif- 
ference, 

(hx(t))x t ,=Px\x(t-t'). (A2) 
Then by the product rule 

j(t) cx d t P x]x {t - r)p xlx (t+ - t) 

= P xlx (At)P xlx (At) - P xlx (At)P xlx (At) 



By consequence both recrossing fluxes 
ish. 



(A3) 

jxx in Eq. 15 van- 



Importantly, this conclusion rests on the three assump- 
tions of (a) a transition region with negligible occupancy, 
(b) a time lag At long enough for any fast internal mem- 
ory of the system to decay, and (c) at the same time short 
enough so that the system state is quasi-stationary with 
respect to any external driving or other slow relaxation 
dynamics. 



Appendix B: Influence of the macroscopic time scale 

Fig. 8 is analogous to Fig. 4b in the main text, but for 
choices of time lag At that are not well separated from 
either microscopic relaxation or external driving; the re- 
crossing flux ]aa is seen to deviate from 0. 
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Figure 8. Variants of Fig. 4b, with At set to 100 (a) or 2 (b). 
In either case, the recrossing flux does not vanish; in (a) the 
force changes noticeably during At, while in (b) the system 
retains memory on the scale At, which is seen clearly in the 
higher harmonic components in jba- Rate functions are not 
given by Eq. 19. 



Appendix C: Solution for the renewal rate equations 

The rate equations 28 in the non-Markovian clock- 
resetting model may be rewritten in Matrix form as 



q(t) 
s(t|0 = 



-fi t S(t|i / )q(* / )di' - 9tS(i)p(0), (CI) 

where 

S A (t\t>) 
S B (t\t') 



q(t) 



«ab(*) 

QBA(t) 



, S(t) = 

and p(i) = 



S A (t) 
S B (t) 

Pa® ' 
P B (t) 



The net probability flux appearing on the left hand side 
of Eq. 28 is then recovered as dtPsit) = —dtPA(t) — 
QAB(t) — QBA(t)- We are concerned with the time- 
homogeneous case only, so that we may write S(t\t') = 
S(t-f). 

The macroscopic pre-history for t < in Eq. CI is 
summarized in the Sp term. More precisely, 



-9tS(i) P (0) 



-d t S(t - t')q(t')dt' 



(C2) 



describes the first-switch propensities of trajectories that 
started in states A, B with probabilities p(0) at t = 
(having arrived at earlier times). In the case that the 
system was held in a stationary state for all times t' < 0, 
it is possible to show that 



§x(t) 



f^Sxi-f'W' 



(C3) 



To write a closed- form solution to Eq. CI, note that 
this is a convolution integral. After Laplace transforma- 



tion (/(a) = 
as 



/(£)) we may rewrite the equation 

(C4) 



q(a) = [-«S(s)+S(i = 0)]q(s) 

+[-aS(a)+S(t = 0)]p(t = 0) 
= [-sS(s) + l]q(s) + [-sS(s) + l]p(< - 0) 



where 
S(t = 0) 



1 
1 



and S(i = 0) 



1 

1 



We may rearrange to give an explicit solution to the 
clock-resetting rate equation in Laplace space, 



q( S ) = [l-l + S S(s)] [1 - *S(*)]p(0). 



(C5) 



Here the transient survival matrix S is either given as 
part of the specification of the problem or may be ob- 
tained from S via Eq. C3 in the case of stationary pre- 
history. 
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The stationary survival matrix S is given as the solu- 
tion of the equation 

d t S(t\t>) = -k(t\t')S(t\t'); S(t'\t>) = 1, (C6) 

where k( t |f ) = [ k ^ k J m ] , (C7) 

which is just given by exp[— j' k(t"|i')dt"] 1. 



Appendix D: Definition of the uninterrupted indicator 
function H 

When writing the function HA(t,t') as a continu- 
ous product, some purely mathematical subtleties arise. 
These are immaterial for the physics of two-state switch- 
ing, and disappear when discretizing time as would effec- 
tively be done in cither experiments or simulation. 

Without restriction, we take the state indicator func- 
tions hx{t) to be right-continuous; i.e. we impose that 
hx(t) — hx{t+) = lim e ^o hx(t + e). This means in par- 
ticular that the time derivatives are pre-point delta func- 
tions at the switching times: h x (t) = ±S(t — t S witch-)- 
It is then convenient to define 



H x (t,t')= J] h x (t"); 



(Dl) 



t>t"># 



this definition makes Hx(t,t') left-continuous in t and 
right-continuous in t': Hx(t-,t' + ) = Hx(t,t'). Then the 
relations Eq. 29 given in the main text, 

d t H x (t,t') = h x {t)H x {t,t') < and 
d t ,H x {t,t') = H x {t,t')hx{t') > 0, (D2) 

are justified. Other conventions for the sample paths are 
possible but may require the insertion of limits in Eq. 29. 
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